A Reference Framework for Multi-Surface Interaction

Physical surfaces are pervasive and serve many purposes. Digital computation is a powerful source of functional support. However, it has been confined to the augmentation of single objects only. In this article, we are interested in the combination of physicality with computation in the context of multiple objects. We propose the notion of multi-surface interaction as a unifying paradigm for reasoning about both emerging distributed UI’s and known interaction techniques such as GUIs, tangible UIs, and manipulable UIs. Multi-surface interaction is expressed within an ontology that shows how our concepts feed into the design of sound foundational software for the development of multi-surface user interfaces.